ndjson

What is ndjson?

The ndjson npm package is used for parsing and stringifying newline-delimited JSON (NDJSON) data. NDJSON is a convenient format for streaming JSON data, where each line is a valid JSON object. This package allows you to easily work with NDJSON data in Node.js applications.

What are ndjson's main functionalities?

Parsing NDJSON

This feature allows you to parse NDJSON data from a readable stream. The code sample reads NDJSON data from a file and parses each line into a JSON object, which is then logged to the console.

const ndjson = require('ndjson');
const fs = require('fs');

fs.createReadStream('data.ndjson')
  .pipe(ndjson.parse())
  .on('data', function(obj) {
    console.log('Parsed object:', obj);
  });

Stringifying NDJSON

This feature allows you to stringify JSON objects into NDJSON format. The code sample takes an array of JSON objects and writes them to a file in NDJSON format.

const ndjson = require('ndjson');
const fs = require('fs');

const data = [
  { name: 'Alice', age: 30 },
  { name: 'Bob', age: 25 }
];

const stringifyStream = ndjson.stringify();
stringifyStream.pipe(fs.createWriteStream('output.ndjson'));
data.forEach(item => stringifyStream.write(item));
stringifyStream.end();

Other packages similar to ndjson

JSONStream

JSONStream is a package for streaming JSON data in Node.js. It can parse and stringify JSON data, but it is more general-purpose compared to ndjson, which is specifically designed for NDJSON format. JSONStream can handle nested JSON structures and provides more flexibility in processing JSON data.

event-stream

event-stream is a collection of stream utility modules for Node.js. It includes functionalities for working with JSON data, such as parsing and stringifying. While it is not specifically designed for NDJSON, it can be used to achieve similar results with more general stream processing capabilities.

split2

split2 is a small utility for splitting streams of data. It can be used to split NDJSON data by newlines and then parse each line as JSON. While it is not as specialized as ndjson, it provides a lightweight alternative for handling NDJSON data.

README

ndjson

streaming newline delimited json parser + serializer. Available as a JS API or a command line tool

usage

var ndjson = require('ndjson')

ndjson.parse(opts)

returns a transform stream that accepts newline delimited json and emits objects

example newline delimited json:

data.txt:

{"foo": "bar"}
{"hello": "world"}

If you want to discard non-valid JSON messages, you can call ndjson.parse({strict: false})

usage:

fs.createReadStream('data.txt')
  .pipe(ndjson.parse())
  .on('data', function(obj) {
    // obj is a javascript object
  })

ndjson.serialize() / ndjson.stringify()

returns a transform stream that accepts json objects and emits newline delimited json

example usage:

var serialize = ndjson.serialize()
serialize.on('data', function(line) {
  // line is a line of stringified JSON with a newline delimiter at the end
})
serialize.write({"foo": "bar"})
serialize.end()

license

BSD-3-Clause